AD/A-002  923 


STRONG  FENCHHL  DUALITY 


A . B e n - T a 1 


Texas  University 


Prepared  for: 

Office  of  Naval  Research 


October  1974 


DISTRIBUTED  BY: 


National  Technical  Information  Service 
U.  S.  DEPARTMENT  OF  COMMERCE 


tSrt'urttv  clanihratton  of  title,  body  of  ab-f  tract  end  indexing  annotation  munt  be  entered  when  the  overall  report  la  c laatified) 


l originating  activity  (Corporate  muthor) 

Center  for  Cybernetic  Studies 
The  University  of  Texas 


1 REPORT  TITLE 


Strong  Fenchel  Duality 


« Descriptive  NOTES  (Typa  ol  tapotl  and.  Indus  tv  data.) 


5 auTMORISI  (Fltal  namt,  middla  Initial,  laal  namt) 


A.  Ben-Tal 


CONTRACT  OR  GRANT  NO 


N00014-67-A-0126-0008;  0009 

6.  PROJEC  T NO 

NR  047-021 


la.  TOTAL  NO.  OF  PAGES  1 7b.  NO.  OF  REFS 


M.  ORIGINATOR'*  REPORT  NUMBERIS) 

Center  for  Cybernetic  Studies 
Research  Report  CCS  200 


•6.  OTHER  REPORT  NO(Si  (Any  other  numbere  that  may  be  aeetgned 
thle  report) 


10  DISTRIBUTION  STA/EMENT 


This  document  has  been  approved  for  public  release  and  sale;  its  distribution 
is  unlimited. 


II-  SUPPLEMENT  ART  NOTES 


12.  SPONSORING  MILITARY  ACTIVITY 


Reproduced  from 
est  available  copy. 


Office  of  Naval  Research  (Code  434) 
Washington,  D.C. 


IS.  ABSTRACT 


Fenchel' s Duality  Theorem  concerns  the  problem  of  minimizing  the  difference 
of  a convex  function  f and  a concave  function  g.  The  duality  resides  in  the 
connection  between  the  above  primal  problem  and  the  dual  problem  of  minimizing 
the  difference  of  the  concave  conjugate  g*  and  the  convex  conjugate  f*.  In 
general  a duality  gap  may  exist  between  the  two  problems  unless  some  regularity 
condition  is  imposed.  Here  a family  of  different  duals  is  suggested  for  which 
a duality  gap  does  not  exist. 


DD  ,"'“.,1473  <PAGE  >» 


S/N  0101-807-681  I 


Reproduced  bv 

NATIONAL  TECHNICAL 
INFORMATION  SERVICE 

U $ Department  ol  Commorce 
Springfield.  VA.  22151 


Unclassified 

Security  Classification 


A-J1408 


DD 


,1473  ( BACK ) 


Unclassified 


S/N  0102*01  4*9400 


Security  CUstificotion 


Research  Report  CCS  200 


STRONG  FSN^I-IEI,  DUALITY 


by 


A.  B.  n-Tal 


October  1974 


Thi  - researc  h vns  partly  supported  by  r reject  No.  NR  047-021, 

ONR  Contracts  N00014-67-A-0 .126-0008  and  Me0014-67-A-0126-0009 
with  the  Center  for  Cybernetic  Studies,  The  University  o Texa^.. 
Reproduction  in  whole  or  in  part  is  permitted  for  any  purpose 
of  the  United  States  Government. 


CENTER  FOR  CYBERNETIC  STUDIES 

A.  charncs,  Director 
Business-Economics  Building,  512 
The  University  o."'  Texas 
Austin,  Texas  78712 
(512)  471-1821 

| PT'TP.'T'C 


l! 


— - 


sSSo1 


ABSTRACT 


Fcnchol 1 „ Duality  Theorem  concerns  the  problem  o'  minimizing 
the  Ji.ffei.cncc  of  a convex  function  f and  a crncn;t  .function  g. 

T.  i’lali. cy  r<  vji.n.  . ->  the  connection  between  the  above  primal 
L/j- ■>.  .in  a"  i Lb,  bu;-V  rroLn.m  of  minimizing  the  Ji'fc  r<  v c o Luc 
concave  conjugate  y*  and  tnc  convex  conjugate  In  general  a 

•Ju -y  gap  may  exist  between  the  two  pror:  1 unless  some 

regularity  ( nrdition  i‘  imposed.  Here  a amily  of  different 


Ju'j ’•  fo  juggee cud  .or  which  a duality  ga^.  do  . n t exist. 


1 


1.  Introduction 

Fenchel's  Duality  Theorem  concerns  the  problem  of  minimizing 
f-g  where  f and  g are  convex  and  concave  functions,  respectively. 
The  duality  resides  in  the  connection  between  minimizing  f~g  and 
maximizing  g*-f*,  where  g*  and  f*  are  the  conjugates  of  g and  f, 
respectively.  More  precisely, 

( + ) inf(f  - g)  =*  max(g*  - f*) 


provided  the  following  regularity  condition  holds: 

The  relative  interiors  of  domain  f and  domain 
' ' g possess  a point  in  common. 

For  treatment  of  Fenchel's  Duality  in  finite  dimensions,  see 
e.g.  [2], [3],  r6],[7],  and  T81,  and  in  infinite  dimensions,  e.g. 
[IT, [4]  and  [51. 

lc  f and  g are  restricted  to  certain  subfamilies  of  convex 
and  concave  functions,  then  (+)  holds  even  without  (*)  being  valid. 
Such  subfamilies  are  the  polyhedral  convex Tand  concave  functions 
f 6 3 , or,  more  generally,  the  stable  functions  [7,  chapter  5]. 

In  this  paper  we  are  interested  in  finding  duals,  other  than 
the  Fcp.chel  dual:  (Sup  (g*  - £*) ) for  which  a relation  similar 
to  (+)  holds  for  every  pair  of  convex  and  concave  functions, 
whether  (*)  holds  or  not.  Such  duals,  called  "strong  Fenchels* 
duals",  are  constructed  in  section  3. 


_ _ 


In  section  4,  the  results  of  section  3 are  applied  to 
Rocka fellar ’ s extension  of  Penchel's  duality  T 6],  and  to  the 
well -known  ioj-mulax  or  computing  the  conjugate  function  and  the 
subdi  'ferentia  1 or  the  sum  of  conv'ex  functions.  A specie!  duality 
result  *cr  a «.  e eta  in  strong  Fenchc  1 dur.3  xs  derived  in  srction  5. 

'’’hr  terminology  used  in  this  paper  is  that  ot  Rocks fe  1 lar ' s 
book  fh”1.  list  below  some  notations  used  in  th~  . eque  1 , for 

• ie  Cin' t ion;-  and  f urth  ■ ->ils  consult  [6,  Part  1 1 . 

L*  t S be  a non'  ro  ty  convex  subset  of  R°,  and  !et  f and  h be 
convex  functions:  Rn -*■  R.  We  denote  by 


ri  S --  the  relative  interior  of  S 
rhd  s — the  relative  boundary  o'  S 
aff  S — the  affine  hull  of  S 
diin  S --  the  dimension  of  £ 

• |x)  — the  indicator  function  of  S 
f Qh  — the  infimal  convolution  of  f and  h,  i.e 
(f  Q h) (x)  = inf ( r (y)  + g(x  - y) ) 

y 


2.  Fenchel*8  Duality 


I,et  $>  be  the  set  of  all  quadruples  (f,  g,  A , A9)  such  t 

1 *■ 


hat 


A and  A^  are  convex  svbsets  of  Rn 


(I) 


f:  Rn  -*■  R i..‘  i proper  convex  function  with  ooir  i.  » * 
g:  Rn  -*■  R is  a proper  concave  function  with  dem  c = A2 
A = Ax  0 A2 


-3- 


Consider  the  primal  problem 


(?) 


in  * ( f - g) 
X^A 


Let  f*  denote  the  (convex)  conjugate  of  f,  and  9*  the  (concave) 
conjugate  of  g,  i.e. 

£*(x*)  = sup(<x?  x>  - ': ( x ) ) 


xe  r 


A 


1 


(;*(>:*)  = in"(<x*  x>  - g(x)) 
xE  A 

2 

. ★ A 


Denote  also  A*  ^ dom  f* , A_  =*  dom  g*  and  finally  A*  = A*  n A*, 
12  i 2 

The  problem 

(P)  ^up  (g*  - f*) 

y*  ea* 

is,  called  the  Fenchel  Dual  of  in. 

The  following  classical  result  relates  ( p) and  IP). 

Fenchel 's  Duality  Theorem  (e.g.[6,  Theorem  31. I1) 

If 


Let  (f,  g,  A , An)e  <J> 
.L 


(2) 


(3) 


then 


ri  A 0 ri  A2  ji  jj 


inf(r  - g)  - max(g*  - f*) 
A A* 


If  f and  g are  closed  and 
(4)  ri  A*  n ri  A*  / 0 


then 


(5) 


min(f  - g)  ■ max(g*  - £*) 
A A* 


□ 
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There  are  well-known  examples,  where  neither  (2)  nor  (4)  holds, 
and  in  which  inf  (P)  > sup  (V)  , i.e.  there  is  a duality  gap. 
One  such  example  is  the  following. 

Sxa  .plc  .1  T7,  p.  181-183j 


’ c c 


ht.n 


= {(>:,  y)  e R2:  x = 0,  v ’ C } 
A0  = <(x,  y)  £ R2:  x * 0,  y ' C } 

(x,y)  e 


f(x,y)  = i 


f*(x*,y*)  = 


y* (x* , y*)  - 


g(*-y)  = \ it?. 


l " 

otherwise 

r l 

i 

(x,y)  e A2 

and  xy 

? 1 

}J<y 

(x,y)  e A2 

and  xy 

< 1 

l - 

otherwise 

f 

y*  < 0 

r 

otherwise 

[-1 

x*  > 0,  y 

* = 0 

- 

otherwise 

Trv  rc  Fore 


inf ( f - g)  = 0 > -1  = sup (g*  - f*)  . 

3.  f.trong  Fenchel  . Duality 
For  any  subsets  B-l<  B^ , of  Rn  such  that 


(6) 


Bi  C Aif 


i = 1,  2 
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let  us  denote  the  followinj: 


f*  (x*)  ■ sup(<x?  x>  - f(x)) 

1 XEBj^ 

g*  (x.)  * inf«xT  x>  - 9<x)) 


xeB- 


B*  ^ dom  f*  , BJ  ^ dom  g*  , B*  - B*  OB*. 


B' 


Also  let  ( p;  B , B ) denote  the  following  problem 
1 2 


( V:  B1#  B2) 


sup(g*  - f*  ) . 
B*  02  B1 


Aheru  ver  = A1#  and  * A2  they  are  omitted  from  t'.e  above 


notation,  thus  .*  = f*.  g* 


g*.  (ft  Aj,  A2)  * (P).  A pair  of 


convex  subsets  (B^ , B^)  is  called  admissible  if  it  satisfies  (6) 


and 


(7) 


B1  " B2  * A 


An  admissible  pair  is  called  strongly  admissible  if  in  addition 
to  (5)  and  (7)  it  satisfies 


(8) 


ri  B1  ^ ri  B2  ft  f/. 


The  following  result  is  an  elementary  observation  suggesting  the 
possibility  of  constructing  duals  ( V;  B^,  B^)  without  duality  gaps. 


Proposition  1 

Let  (f,  g,  A1#  a2)  e 4>  and  let  (B^,  B ) be  an  admissible  pair. 


Then 


inf  ( f - g)  * sup(g*  - f*  ) a sup(g*  - f*) 
A B*  B2  B1  A* 


(9) 
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Proo 


From  the  dt;jnii  iotv  of  f*  and  g*  we  derive 


Bi 


'B- 


* 

(x*) 

« 

*' 

X 

V 

x> 

- (X)  x e 

B . 

x*  e 

B1 

1 

g* 

(x*) 

i 

cx* 

x> 

i 

X 

X 

r> 

B . 

x*  c 

B2 

2 

2 

ncc  ror  every  x 

£ B! 

n 

B2  ' 

= A 

and  x*  e B*, 

* 

1 


«■*  ( ••:*)  + ‘ (x)  § < x*  x > S * (x*)  ♦ f( -■) 


- ,(x'  * g*  (x*)  - f*  (x*)  x e A,  x*  e B* 


B 


*2  B1 

oro’irc.  thnt  ‘irst  inequality  in  (9).  To  ;.>rove  the  second 
inequality  note-  that 


B C A ^ 
1 


S * f* 
* 

Bt  * \ 


fg;  * g* 

»2r\*4  1 . 

lS*  = Bi 


>:.:n  * ti  nt. 


i* 

B 


_ f 


n 


S g*  - f* 


B*  =>  A* 

'rom  *hjLth  it  follows  that 


>up(g*  - f*  ) s sup(g* 
B*  B'  B1  A* 


- *) 


a 
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A 1 problem  (t>;  Bj,  B2)  is  called  a Btrong  Fenchel  dual  if 

(10)  inf  ( P)  - max(p;  B2> 

lor  every  (f,  c; , A , A ) e ♦ . This  property  is  closely  related 

1 2 

to  the  stroncj  admissibility  of  (B^,  Bj)  * as  expressed  in 
Proposition  2 

For  every  strongly  admissible  pair  (B^,  B2) # the  problem 
(P;  B, . B ) is  a strong  Fenchel  dual. 

Prop 

Thfc  admissibility  of  (B^,  Bj)  implies 

(11)  inf  ( * - g)  - inf ( f - g) 

A B1°B2 

where 

f - f + 6(  lB1) 
g ^ g - «( |B2) 

clearly 

A * 

u,  g,  B1#  B2)  e * 

moreover,  by  the  strong  admissibility  of  (B^ , B2)  it  follows 

£ rom  Fenchel ' s Duality  Theorem  that 

(12)  inf(f  - g)  * max(g*  - f*) 

B ^b 
1 2 

but 

f*  ■ f*  , g*  ■ g* 

B1  b2 


-8- 


hence  (11)  and  (12)  implies 


inr(-c  - g)  * max(g*  ■ En  > 

A B*  B2  B1 

The  existence  of  a strongly  admissible  pair,  i.e.  the  nonemptiness 

of 

(13)  S = {all  strongly  admissible  pairs} 

is  illustrated  by  the  following  simple  example. 

Example  2 

Consider  the  pair 

B » A,  B2  = A 

then 


n 


n = a c a.  , b2  * a c a2,  b]  o b2  « a ^ a = a 

ri  ri  B2  = ri  A 0 ri  A = ri  A / ff 


hence  (A,  A)  in  strongly  admissible.  The  fact  ri  A ^ indeed 

holds  (in  finite  dimension  spaces) for  any  nonempty  convex  set  A. 

Note  that  Example  2 together  with  Proposition  2,  produce  our 

■ irst  strong  duality  relation 

ini  (l'  - g)=  max(g*  - f*)  . 

* A A 


4.  Characterization  of  strong  admissibility 


Lemma  1 

For  any  nonempty  con  /ex  sets  S,  T c Rn 


-9- 


(14)  T 0 ri  S * f( 

i :,  and  only  if 

(3  5)  S n t n rbd  S 


consequently 

(16)  ri  S 0 ri  T * J2f 


if,  and  o ly  i " 

(17)  [sot  rr  rbdS  ] V [S  H T c rbd  T] 


Prop  f 

First  note  that  the  equivalence  (16)<s^(17)  follows  from  the 
equivalence  (14)«j=^  (15)  since 

L-'i  S 0 ri  T = 0]<=>[T  n ri  S - 0]v[S  n ri  T « 0] 

Indeed  the  implication  (<=  ) is  trivial,  and  the  implication  ( =£) 
follows  from  the  fact  that  the  condition  ri  S 0 ri  T “0  ^is 
necessary  and  sufficient  for  proper  separation  of  S and  T (see 
[6,  Theorem  11.3]).  Now,  if  S H T ■ 0,  the  equivalence  (14)  <=>(15) 
is  trivial. 

Thus  suppose  that 
(IB)  f n T / 0 

Let  (14)  hold.  Then 

(19)  (S  0 T)  0 ri  S - (S  n ri  S)  n T - T 0 ri  S « 0 


-10- 


..inrt  (r  r T)  /t  S it  follows  from  (19)  that 

£ OTtS-riScclS-riS*  rbdS. 

Suppose  now  that  (lri)  holds.  Clearly  ' rbdS  o ri  S = 
hcnoi,  by  ( 5)  , (£  o t)  n ri  S = 0 and,  by  (19)  , T *">  ri  L’  = 0. 

□ 

Corollary  1.1 

The  set  S of  all  strongly  admissible  pairs  is  given  by 
(20)  5 = {convex  pairs  (B.  , B-)  : A c B.  r A.  , A rbJ  B.  , i = 1,2} 

1 ^ X * i 

Prop  F from  Lemma  1 

ri  n ri  B2  ^ 0 B ^ D B 2 £ rbd  B^  i = 1,  ~ 

Nov;  Li  K;  L'act  [A  <“  Ft  r A ^ , i * 1,  2l  is  equivalent  to 

o B2  = A,  B.  c A^  i = 1,  2]  and  hence  the  result. 

1 □ 

The  following  lemma  will  enable  us  to  find  an  important  subset  of  S. 

Lemtia  2 

For  any  non-empty  ronvex  sets  S,  T rr  Rn  and  any  convex  subsets 
P,  O such,  that 

( ')  r o t - p r r n aff(s  n ") 

(22)  5 n T - 0 - T 0 aff(S  0 T) 




r 

it  tol lows  that 

(23)  ri  M ri  Q / ((. 

Prop 

It:  (^3)  is  false  then 

[Q  n ri  P = 0]  V [P  D ri  Q = 01. 

Thu  , without  ':)bS  O'  generality,  suppose  that 
(..’4)  Q 0 ri  P = 0 

Thi..  is  cqui /?' i an  t , by  .emma  ] to 

p 0 Q r-  rbd  P . 

since  this  means  that  p fl  Q is  a convex  sufc.  et  o the  relative 
boundary  o ' the  convex  set  P it  follows  [Corollary  6.3.3]  that 
(25)  dim(r  n Q)  < dim  P. 

On  thf  other  hand  (21)  and  (12)  imply 

C n T #-  p 0 Q c (f  * T)  n a rf  (f  o T) 

•*.  . ?■  (. 

( /.h)  s h T = P o o 

Morco /er 

dim  P £ dim  [P  0 aff(S  o T)  1 5 

« dim  at:f(S  n T)  ■ dim(S  o T) 

[I  n.-  by  (.’<>) 


dim(P)  = Jim(P  0 Q) 


ntrndictinq  (2r>) 


P n ri  Q ^ 0 


Pro^/inc  (2  3)  . 

P 

’c.tatinq  I tiTiiia  2,  v..  obtain 
r L i nv  2 . . 


c t 


(;;7)  A = (convex  pairs  (B],B2):  A c Bi  c arf  A,  i=  , 2} 

con;  i_ts  o tromjjy  admissible  pairs,  i.e.  A CS* 


Remarks 

i.  A necessary  condition  'or  (B^ , B2)  to  be  strongly  admissible 

r i A " r i P , i = 1 , 

L 

Th_  -j  ■ ] o £i  rf  m thf  at  ion  (see  rA,  Coro)  !ary  ‘.5.23): 

A " 

^ ri  A c ri  B. 

A rbd  B;  1 

^h'-rc  arc  pairs  (Aj,  A.,)  Tor  which  A = S,  such  as  the  pair 
/A  A ) ni'/'-n  in  Fxamolr  1 . 

Th<  re  are  of  course  • c ts:  ( ^ -j  , A2)  for  which  A jl  S . Consider 
•••(  !'  A i = • , K in  the  plane.  A.,  - a ; • iiui  o'  the 

J ci-.  Thin  (B,,  it)  « (A1(  a2)  C s but  (R,,  IV,)  j:  A. 


■uMlIMt 


1 . I 
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A20  ri  A 1 1*  0 


then  the  ^et 

A 1 * { (B1  , B2)  : A <-  Bl  r A1#  A r-  B2  C A^  aff  A} 
(.ontainud  in  S (and,  clearly,  contains  A).  This  fact  follows 
actually  * rom  the  proof  of  Lemma  2. 

A.  Some  related  results. 


Roc;  c ?.  ) a r ' : ■ xtnn.  ion  oi  Fenchel  duality 

Suppose  Ld 

f C . i:.  a nonempty  convex  subset  or  Rn 


(?P) 


1 

C„  is  a nonempty  convex  subset  o c R 

4 

n 


m 


f:  R R i 5 a proper  convex  function,  dom  f » 
g:  Rm  -*■  R is  a proper  concave  function,  dom  g ■ 

U:  R RlT1  ir  a linear  transformation  with  inverse  u~ 

v.i.i  re,  for  any  f <”  Rm 

M“'  S - { X:  Mx  E S)  . 


cuppose  further  that 

(?9) 
anu 


^ -1 
C » C2  o M C2  ^ 0 


(30) 


C0  r Range  m 
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I.et  the  set  of  all  ( r,  g,  fc,  C1#  C?)  satisfying  (28)  - (30)  be 

lenotc  'i  by  t . 

Con.-idcr  uie  problem 

(31)  inr(f  - .ifl) 

The  in.imum  is  taken  effectively  on  the  (nonempty,  convex)  set  C. 
Mote  that,  there  is  no  loss  of  generality  in  a.  suming  (30),  for 
if  (f.  9.  M,  £ ^ , T\.j)  satisfy  (28),  (29)  but  not  (30),  one  can 
con. i^er  instead  o'  (31)  the  equivalent  problem 

(32)  inf{(f  - gM)  : xe  0 M 1C0) 

where 

_ ^ 

g = g - 6(  I range  M) 

and 

* 

C * dom  a 
2 

Clearly  then 

( f , g,rt,  C , C2)  e Y. 

Rocka  ellar  (see  e.g.  [6.  Cor  >llary  31.2.1]  and  [71)  proves  that 
i'  ( , g,  M.  C} , c: ) e Y and 
( ")  ri  C,  c m- ^ ( i i C2)  ^ 0 

Then 

(3'-)  inf(f  - gM)  * Max(g*  - f*M*) 

where  M*  is  the  adjoint  of  M. 


-15- 

Tln-  i>  . nib  o'  tlie  previou  section  can  be  u.ed  here  to  derive 
the  olJov.  my. 


T.icofcn  1 

Let  (f,  c , M,  C ^ , C2)  e V,  and  let  c Rn,  D2  C Rm  an^ 

con/rx  ubsets  sati-  ‘ying 

(35)  C c Dx  n C1,  C c if1!)  c M-1C? 

(36)  C £ rbd  D 


(37)  C t M“ 1 ( rbd  D2) 

Then 

( 'R)  in.(f  - ' M)  * max  (g*  - r* 

l>2  d 

In  >,•  i:tic  a J ar  , (3R)  hollo  i* 

C c Dx  <-  aff(C)  ft  C1 

(39) 

C C *-  aft'(C)  ft  k“1C2 


in  .'  ( f - ' M)  * max(g*  - I*  M*) 

d2  0^ 


Proo " 


First  we  collect  : one  properties  of  n”  ^ needed  below. 


(40) 

S n T =^>M_1S 

r-  m_1T 

(41) 

M-.1  (s  r t)  = 

M”1?  ft  M-L 

(42) 

M~J  (S  ~ T)  = 

M_1S  ~ M" L 

(43) 

Range  M to  S, 

S ^ ^ = >M 

Finally  (see  e.ct.rfc.  Theorem  6.  73) 
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(44)  fri(M-1S)  - M-1(ri  f) , 

M_;L(ri  S)  0 0 4/ 

[fl  (M_1S)  - M_1(cl  S)  . 

Now,  by  (42)  , (37)  i;;  equivalent  to 

(45)  C (f  M"  ((  ' D,J  ~ M-1(ri  D2)  , 

?\  ! 6 O 

0 ? ri  r D.  r c - Range  M,  by  (35)  and  (30) 

Hence,  by  (43) 

(46)  M“l  (>'  D.,)  jt  0 
onJ  Lhu.,  (44)  - (46) 

(47)  C 0 cl(M"LDr)~  ri(M_1D2)  = rb](M~ 'D  ). 

It  '0.1 1 that  f-  * 01  and  T = M~^D0  are  two  subsets  ol  Pn 
ati  'ying  V.  " t = C (by(35))and  SOT/  rbd  5-  , £ n t 0 rbd  T 
and  hence,  by  Lemma  l,  ri  E ri  T / 0,  or  in  view  of  (46)  and 
(44)  : 

(48)  ri  D|  0 M-1  ri  D2  0 0. 

Fx'om  (35)  it  follow.,  that 

A A 

int(f  - gM)  * inf(f  - gM) 

where 

f & f + 5(  |DX) 

<:  * g - $(  !d2)  . 

A A 

(F,  g,  M,  Dj , .0, ) e * 


Clear  ! y 
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and  hence  (38)  follows  from  the  validity  of  the  regularity 
oml  it  ion:..  (48).  Finally,  (39)  implies,  by  lemma  2,  that 
ri  D ^ ri  m"1^  / 0 whicn,  again,  by  (46)  and  (44)  , implies 
(48)  , pro/ina  the  last  assertion  of  the  theorem. 


Th  c n ju-jate  of  the  : nm  of  convex  functioru- 


Thcorem  ? 

l et  ( f , -h,  A1 , A2)  c 4> 
attained  and 


Then,  the  infimum  in  f*  p h*  is 

"1  °2 


(49)  (f  + h)*  - f*  ah* 

B1  B2 


o>  .very  (B^ , B2)  e S (sec  (20))  and,  in  particular,  for  every 
^ , B2)  e A ( ce  (27)) 

rroo  ■ 

(f  + h)*(y*)  **  sup(<y*,  x>  - f'(x)  + h(x)  1) 

* -in "( f (x)  - [ <y* , x>  - Mx)]) 

••  -inf  ( f (x)  - n(x) ) 

where 

g(x)  = < y*,  x > — h(x) 


P 


Now,  ( r , <: , A , ’'Jet  , <">nd  (B.,  B ) are  stronjly  admissible 

•-  1 « 

(C  (ire  I Laric:.  I .1  and  . 1 ) .hence  by  proposition  2 

-in  T( f - g)  ■ -max(g*  - r*  ) 

°2  Bj 


A simple  calculation  ^hows  that 

9*  (**)  = -h*  (y*  - x*) 

Br;  b2 

5.0 

(f  + h)*  (y*)  ■ - inf(f  - g)  * -max (9*  - r*  ) » 

B2  b] 

= -max(-h*(y*-  x*)  - f*  (x*))  = 

B B1 

= min ( * (x*)  + h*  (y*  - x*) ) = ( '.*  p h*  ) (y*) 
bl  b2  B1 

Theorem  2 generalized  [b , Theorem  16.41.  The.  "in  ima  1 con- 
volution formula"  ('>'?)  wa:  first  obtained  vu  Fenchel  [?1.  See 

also  [7], 

The  subdifferential  of  the  sum  of  convex  functions 

Let  f be  a convex  function,  and  S a subset  of  dom  f.  Consider 
for  x e S,  the  set  >cf(x)  off  all  x*  e Rn  such  that 

f(z)  > f ( x ) +<x*,  z — x > , V z e f 

'h  wrice  *f(x)  for  ^jom  f(x),  thus  actually 

(X)  = >(  f(x)  + 6(x[s) ) . 

Theorem  3 

j^et  (f,  -h.  A,,  A.J  e $ . Then 

1 c 

(50)  * ( i + h)  - >Bjf(x)  + 

or  > yery  (B^ , B2)  e S nnd,  in  particular,  for  e/cry  (B^,  B„)  F ^ 


where 
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*<£  + h)  - VlP  A2(£  ♦ h)  - B2(f  * h> 


£ « f + « iBj) 

A 

h = h 4 $ |b  ) 


the  la^t  « quality  is  justified  by  the  fact  that  (Bj,  B2)  arf 

* A 

admissible.  Now,  since  (f,  h,  , B2)  e ♦ and  ( , n.,)  are 
trongly  admissible,  it  follows  that  (see  [6,  Theorem  23.81 


but  *f  = >_  f 
B1 


'b^o  B2(f  + h)  ■ *f  + *h 

A 

*h  ■ h and  hence  (50)  follows. 
b2 


5.  A special  result  for  the  strong  Fencheis*  dual  (P;  A,  A) 


It  was  shown  in  Example  2 that  (A,  A)  is  a strongly  admissible 


pair,  and  hence 


inf(£  - g)  « max(g*  - f*)  . 
A A A 


The  following  theorem  adds  to  the  validity  of  (50)  an  explicit 
connection  between  the  optimal  solutions  of  (p)  and(p;  A,  A). 

The  proof  does  not  rely  on  Fenchel's  Duality  Theorem,  or  its 
traditional  proofs(  e.g.  [61,  [7],  and  [4})  and  in  fact  does  not 
utilize  separation  arguments.  This  is  significant  in  deriving 


generalizations  of  (5l)  for  nonconvex  functions. 
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Theorem  4 

L-  1.  (r,  n,  A?)  e (f,  and  suppose  further  that  r,  g e C *. 

Let  xeA  be  an  optimal  solution  of  (P)  . Then  any  x*  belonging 
to  the  interval 

[Vf(x),  Vyx: 

i.  • n optimal  .olution  o'  (V  ; A,  A)  and  (51)  is  yalid. 
rroo 

Since  f h convex  on  A ^ , it  satisfies  th  gradient  inequality 
. (x)  > -(x)  + <x  - x,  7 f (x)>  x e A^ 

and  hence,  in  partic. lar 

(52)  f*(7f(x))  - <Vf (x)  , x>  - f(x)  ft  <7f(x)  , x>  - f(x),  xe  A 
and 

(53)  <Vf(x) , x - x > f f(x)  - f(x) , xeA. 

*.  nei.^  sary  condition  i:or  x to  solve  (?)  is  (see  e.g.  [4,  Theorem  2, 

...  17  51) 

<Vf  (o  - V f)  (x)  , x - x>  * 0 XEA 

or  rr  mginy  term: 

( 54)  <Vy  (x)  , x - x > * <\Df(x) , x - x > x e A 

(53)  with  (54)  imply 

<7!(x),  x - x>*  f(x)  - r(x) 


§ 
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or 

<7<j(x),  x>  - f (x) . * <79  (x)  , x > — f (x)  x £A 

i .e . 

(55)  f*(V9(x)  = <7g(x),  x>  - v(x)  . 

A 

But,  • imi lar  to  (5i), 

(i6)  3*(Vg(x))  ■ <79 (x)  , x>  - g(x). 

Now,  (55)  and  (56)  show  that 

(57)  f(x)  - g (x)  - g*(Vg(x))  - f*(7g(x)). 

A A 

Since  (see  Proposition  1) 

f(x)-  y(x)  * 9j^(x*)  - f*(x*)  for  every  s* 
it  oilows  from  (56)  that  x*  » 7g(x)  is  an  optimal  solution  o' 

( ft  A,  A),  and  that  (5l)  is  valid. 

Similar  to  (57),  it  can  be  shown  that 

f(x)-  g(x)=  g*(7  f (x) ) - f*(7f(x)) 

A A 

which  proves  that  x*  = Vf(x)  is  also  an  optimal  solution  of 
(ft  A,  A)  . 

Finally,  (ft  A,  A)  being  a concave  program  implies  that  its 
.oJution  • et  i~  convex,  and  hence  every  x*  e [yf(x),  vg(x)]  is 


n optima]  solution. 


□ 
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Corollary  4.1 

Dual  program  (P;  A,  A)  has  a unique  optimal  solution  only  ij. 
primal  problem  (?)  has  an  optimal  solution  which  is  a critical 
point  of  its  objective  function* 

Prop 

Let  x be  an  optimal  solution  of  (?) . If  (P;  A,  A)  has  a 
unique  maxifoizer,  it  follows  from  Theorem' 4 that  Vf(x)«  Vg(x) 
i.e.  V ;(x)-  Vg(x)  = C,  hence  x is  a critical  point. 

□ 

Theorem  1 is  illustrated  in  the  following 
Example  3 

Let  f and  g be,  respectively,  a strictly  convex  and  a strictly 

concave  functions:  R -►  R such  that  f - g is  strictly  monotone. 

Let  dom  f 0 dom  g = [a,  b]  (a  < b) . Clearly  then 

min(f  - g)  * f(a)  - g(a) 

[a,bl 

and,  furthermore,  f is  strictly  increasing,  g‘  is  strictly 

decreasing  and  ‘ ' > g'.  Hence 

(58)  g'(b)  < g'(a)  < f • (a)  < f ' (b)  . 

Let  L^  denote  the  Legandre  transform  of  h,  i.e. 

Lh(x*)  = <x*,  h,-1(x*)>  - h(h'"1(x*) ) 


By  the  calculus  then 


ax*  - £(a) 


(59)  f*(x*)  = 
A 


< Lf(x*) 
bx*  - f (b) 

k 


-«  < x*  « (a) 

f 1 (a)  s x*  * (>  ) 

f ' (b)  « x*  < <* 


(60)  g* (x*) 
A 


bx*  - g(b) 
Lg(x*) 
ax*  - g(a) 


-»  < x*  * g ' (b) 
g'  (b)  * x*  «:  (a) 

g ' (a)  ■*  x*  < oe 


Combining  the  information  in  (58)  - (60)  we  derive  the  graphical 
representation  of  the  dual  objective  function  ^ - f*  (see  figu*  •»  1) 
fr  m which  the  conclusions  ofx?Theorem  1 arc  evident. 


FIGURE  1 
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